Data processing device

ABSTRACT

A data processor according to the present invention includes receiving sections  100, 102  for receiving video data and audio data, a compressing section  101  for generating encoded data, complying with the MPEG-2 system standard, by encoding the video and audio data received, an auxiliary information generating section  103  for generating auxiliary information, which includes reference information to make reference to the encoded data and attribute information that uses a VOBU of the encoded data as a sample unit and that describes an attribute of the sample unit, and a writing section  120  for writing the encoded data and the auxiliary information on a storage medium as a data file and an auxiliary information file, respectively. The encoded data is decodable by either the auxiliary information file or the MPEG-2 system standard.

TECHNICAL FIELD

The present invention relates to a data processor and processing methodfor writing stream data of a moving picture stream on a storage mediumsuch as an optical disc.

BACKGROUND ART

Various types of data streams have been standardized to compress andencode video data at low bit rates. A system stream compliant with theMPEG-2 system standard (ISO/IEC 13818-1) is known as one such datastream. There are three types of system streams, namely, a programstream (PS), a transport stream (TS) and a PES stream.

Recently, another data stream, complying with the MPEG-4 system standard(ISO/IEC 14496-1), has been being defined. In a format compliant withthe MPEG-4 system standard, video streams, including MPEG-2 or MPEG-4video streams, and various types of audio streams are multiplexedtogether, thereby generating moving picture stream data. Furthermore,according to a format compliant with the MPEG-4 system standard,auxiliary information is defined. The auxiliary information and a movingpicture stream are defined as a single file (which is called an “MP4file”). The data structure of an MP4 file is based on, and an extensionof, a QuickTime® file format of Apple Corporation. It should be notedthat as for a system stream compliant with the MPEG-2 system standard,no data structure storing the auxiliary information (such as accessinformation, special playback information and recording date) isdefined. This is because the auxiliary information is included in asystem stream according to the MPEG-2 system standard.

Video data and audio data would often be recorded on a magnetic tape inthe past. Recently, however, optical discs such as DVD-RAMs and MOs haveattracted much attention as storage media that will soon replacemagnetic tapes.

FIG. 1 shows a configuration for a conventional data processor 350. Thedata processor 350 can read and write a data stream from/on a DVD-RAMdisc. The data processor 350 receives a video data signal at a videosignal input section 300 and an audio data signal at an audio signalinput section 302, respectively, and sends them to an MPEG-2 compressingsection 301. The MPEG-2 compressing section 301 compresses and encodesthe video data and audio data in accordance with the MPEG-2 standardand/or the MPEG-4 standard, thereby generating an MP4 file. Morespecifically, the MPEG-2 compressing section 301 compresses and encodesthe video data and audio data in accordance with the MPEG-2 Videostandard to generate a video stream and an audio stream. Thereafter, theMPEG-2 compressing section 301 further multiplexes these streamstogether in accordance with the MPEG-4 system standard, therebygenerating an MP4 file. In this case, a writing control section 341controls the operation of a writing section 320. In accordance with aninstruction given by the writing control section 341, a continuous dataarea detecting section 340 checks the availability of sectors beingmanaged by a logical block management section 343, thereby detectingphysically continuous unused areas. Then, the writing section 320 getsthe MP4 file written on the DVD-RAM disc 331 by a pickup 330.

FIG. 2 shows the data structure of an MP4 file 20. The MP4 file 20includes auxiliary information 21 and a moving picture stream 22. Theauxiliary information 21 is described by an atom structure 23 definingthe attributes of video data, audio data and so on. FIG. 3 shows aspecific example of the atom structure 23. In the atom structure 23, thedata size (on a frame basis), the address of the data storage location,a time stamp showing the playback timing and other pieces of informationare described for each of the video data and audio data. This means thatthe video data and audio data are managed as individual track atoms.

In the moving picture stream 22 of the MP4 file shown in FIG. 2, thevideo data and audio data are each arranged on a frame basis, therebymaking up a stream. For example, if the moving picture stream has beenobtained by the compression coding method compliant with the MPEG-2standard, then a plurality of GOPs are defined for the moving picturestream. A GOP is a unit for a collection of video frames including anI-picture, which is a video frame that can be read by itself, and P- andB-pictures that are interposed between one I-picture and the nextI-picture. In reading an arbitrary video frame of the moving picturestream 22, first, a GOP including that video frame is identified in themoving picture stream 22.

It should be noted that a data stream with a structure including amoving picture stream and auxiliary information as in the data structureof the MP4 file shown in FIG. 2 will be referred to herein as an “MP4stream”.

FIG. 4 shows the data structure of a moving picture stream 22. Themoving picture stream 22 includes a video track and an audio track, toeach of which an identifier TrackID is added. Not every moving picturestream includes one track apiece but the tracks may sometimes bechanged. FIG. 5 shows a moving picture stream 22 in which the tracks arechanged on the way.

FIG. 6 shows a correlation between a moving picture stream 22 andstorage units (i.e., sectors) of the DVD-RAM disc 331. The writingsection 320 writes the moving picture stream 22 on the DVD-RAM disc inreal time. More specifically, the writing section 320 secures a logicalblock, which is physically continuous for at least 11 seconds whenconverted at the maximum write rate, as a single continuous data areaand sequentially writes video and audio frames there. The continuousdata area consists of a plurality of logical blocks, each of which has asize of 32 kilobytes and to each of which an error correction code isadded. Each logical block is further made up of a plurality of sectors,each having a size of 2 kilobytes. The continuous data area detectingsection 340 of the data processor 350 detects again the next continuousdata area when the remainder of the single continuous data area becomesless than 3 seconds, for example, if converted at the maximum writerate. When the single continuous data area is full, the continuous dataarea detecting section 340 writes the moving picture stream on the nextcontinuous data area. The auxiliary information 21 of the MP4 file 20 isalso written on the continuous data area that has been secured in asimilar manner.

FIG. 7 shows how the written data is managed by the file system of theDVD-RAM. In this case, either a file system compliant with the universaldisk format (UDF) standard or a file system compliant with ISO/IEC 13346(Volume and File Structure of Write-Once and Rewritable Media UsingNon-Sequential Recording for Information Interchange) may be used. InFIG. 7, the continuously written MP4 file is stored under the file name“MOV0001.MP4”. The file name and file entry location of this file aremanaged by a file identifier descriptor (FID). The file name is definedas MOV0001.MP4 in the file identifier, while the file entry location isdefined by the top sector number of the file entry in the ICB.

It should be noted that the UDF standard corresponds to the installingterms of the ISO/IEC 13346 standard. By connecting a DVD-RAM drive to acomputer such as a PC by way of a 1394 interface and a serial busprotocol 2 (SBP-2), the PC can also treat a file that was written in aUDF compliant format as a single file.

y using allocation descriptors, the file entry manages the continuousdata areas (CDAs) a, b, c and the data area d where the data is stored.More specifically, if the writing control section 341 finds a defectivelogical block while writing the MP4 file on the continuous data area a,then the writing control section 341 will skip that defective logicalblock and continue to write the file from the beginning of thecontinuous data area b. Next, if the writing control section 341 finds anon-writable PC file storage area while writing the MP4 file on thecontinuous data area b, then the writing control section 341 will resumewriting the file from the beginning of the continuous data area c. Onhaving written the file, the writing control section 341 will write theauxiliary information 21 on the data area d. As a result, the fileVR_MOVIE.VRO is made up of the continuous data areas a, b, c and d.

As shown in FIG. 7, the beginning of the data to be referenced by theallocation descriptor a, b, c or d matches with the top of itsassociated sector. Also, the data to be referenced by every allocationdescriptor a, b or c, except the last allocation descriptor d, has adata size that is an integral number of times as large as that of onesector. Such a description rule is defined in advance.

In playing back an MP4 file, the data processor 350 retrieves andreceives a moving picture stream by way of the pickup 330 and a readingsection 321 and gets the stream decoded by an MPEG-2 decoding section311, thereby generating a video signal and an audio signal, which areeventually output through a video signal output section 310 and an audiosignal output section 312, respectively. Reading the data from theDVD-RAM disc and outputting the read data to the MPEG-2 decoding section311 are carried out concurrently. In this case, the data read rate isset higher than the data output rate and is controlled such that thedata to be played back does not run short. Accordingly, if the data iscontinuously read and output, then extra data can be obtained by thedifference between the data read rate and the data output rate. By usingthat extra data as the data to be output while data reading isdiscontinued by pickup's jump, continuous playback is realized.

Specifically, supposing the rate of reading the data from the DVD-RAMdisc 331 is 11 Mbps, the maximum rate of outputting the data to theMPEG-2 decoding section 311 is 8 Mbps and the longest time it takes tomove the pickup is 3 seconds, data of 24 megabits, which correspondswith the amount of data to be output to the MPEG-2 decoding section 311while the pickup is moving, is needed as the extra output data. Tosecure this amount of data, the data needs to be read for eight secondson end. That is to say, the continuous reading needs to last for theamount of time that is obtained by dividing 24 megabits by thedifference between the data read rate of 11 Mbps and the data outputrate of 8 Mbps.

Accordingly, while the continuous reading is carried out for eightseconds, data of 88 megabits, which should be output in eleven seconds,is read out. Thus, if a continuous data area with a size correspondingto at least eleven seconds is secured, then continuous data playback canbe guaranteed.

It should be noted that several defective logical blocks may be includedwithin the continuous data area. In that case, however, the continuousdata area needs to have a size corresponding to an amount of time thatis slightly longer than eleven seconds with the expected read time,which it will take to read those defective logical blocks during theplayback operation, taken into account.

In performing the process of deleting a stored MP4 file, the writingcontrol section 341 performs predetermined deletion processing bycontrolling the writing section 320 and reading section 321. In the MP4file, the auxiliary information includes presentation timings (i.e.,time stamps) of all frames. Accordingly, in partially deleting anintermediate portion of a moving picture stream, only the time stamps inthe auxiliary information need to be deleted. It should be noted that inan MPEG-2 system stream, the moving picture stream should be analyzed toensure continuity even at the partially deleted portion. This is becausethe time stamps are dispersed over the stream.

The MP4 file format is characterized by storing the video frames oraudio frames of a video/audio stream as a single set. At the same time,the MP4 file format defines access information, which enables randomaccess to any arbitrary frames, for the first time ever for anyinternational standard defined so far. The access information is definedon a frame-by-frame basis, and may include the frame size, frame period,and address information for a frame. More specifically, the accessinformation is stored for every unit (e.g., every display period of 1/30second for a video frame and every 1,536 samples for an audio frame (inAC-3 audio, for example)). Accordingly, if the presentation timing of avideo frame needs to be changed, only the access information thereofshould be changed, and the video/audio stream does not always have to bechanged. The access information of that type has a data size of about 1megabyte per hour.

As to the data size of the access information, according to “DVDStandard Part III for Rewritable/rerecordable discs: Video RecordingStandard Version 1.1”, issued by DVD Forum, VR4, pp. 31-35, the accessinformation compliant with the DVD Video recording standard needs tohave a data size of 70 kilobytes per hour. The data size of the accessinformation as defined by the DVD Video recording standard is less thanone-tenth of that of the access information included in the auxiliaryinformation of an MP4 file. FIG. 8 schematically shows a correlationbetween the field names used as the access information compliant withthe DVD Video recording standard and pictures represented by the fieldnames. FIG. 9 shows the data structure of the access information shownin FIG. 8, the field names defined for the data structure, and theircontents and data sizes.

Also, the optical disc drive disclosed in Japanese Patent ApplicationLaid-Open Publication No. 2001-94933 (see FIG. 3, among other things)not only writes video frames on a GOP basis, not on a frame basis, butalso writes each audio frame continuously for a period of timecorresponding to one GOP. The optical disc drive defines the accessinformation on a GOP basis, too, thereby cutting down the required datasize of the access information.

Furthermore, the MP4 file describes a moving picture stream inaccordance with the MPEG-2 Video standard but is not compatible with asystem stream as defined by the MPEG-2 system standard. Thus, the MP4file cannot be edited by utilizing the moving picture editing capabilityof any application used extensively today on PCs, for example. This isbecause the editing capability of a lot of applications is targeted on amoving picture stream compliant with the MPEG-2 system standard.Furthermore, the MP4 file standard defines no decoder model to ensureplayback compatibility for a moving picture stream portion. Then,absolutely no piece of software or hardware compliant with the MPEG-2system standard, which has circulated very widely today, can be used atall.

DISCLOSURE OF INVENTION

An object of the present invention is to provide a data structure, ofwhich the access information has a small data size and which can be usedeven in an application designed for a conventional format, and alsoprovide a data processor that can perform processing based on such adata structure.

A data processor according to the present invention includes: areceiving section for receiving video data and audio data; a compressingsection for generating encoded data, complying with the MPEG-2 systemstandard, by encoding the video data and audio data received; anauxiliary information generating section for generating auxiliaryinformation, which includes reference information to make reference tothe encoded data and attribute information that uses a video object unit(VOBU) of the encoded data as a sample unit and that describes anattribute of the sample unit; and a writing section for writing theencoded data and the auxiliary information on a storage medium as a datafile and an auxiliary information file, respectively. The encoded datais decodable by either the auxiliary information file or the MPEG-2system standard.

In one preferred embodiment, the reference information represents thefile name and storage location of the data file stored on the storagemedium.

In another preferred embodiment, the compressing section generates theencoded data as a plurality of sets, and the auxiliary informationgenerating section generates the reference information that makesreference to the respective encoded data.

In another preferred embodiment, the compressing section generates theencoded data as a plurality of sets, and the auxiliary informationgenerating section generates stream data as a single stream by arrangingthe plurality of sets of encoded data as a series, and also generatesauxiliary information that further describes location informationspecifying the storage location of the encoded data if the data size ofthe encoded data is not constant every time the data is read.

In another preferred embodiment, the compressing section generates theencoded data as either an MPEG-2 program stream or an MPEG-2 transportstream.

In another preferred embodiment, the auxiliary information generatingsection describes an audio frame of encoded audio data, representing theaudio data of the encoded data, as another sample unit in the attributeinformation.

In another preferred embodiment, the compressing section generatesfirst, second and third data files, and the second data file includesframe data that is needed to decode the encoded data of the first andthird data files continuously with no time gap left.

In another preferred embodiment, the auxiliary information generatingsection generates an auxiliary information file that is described in theMP4 format.

In another preferred embodiment, the auxiliary information generatingsection generates an auxiliary information file that is described in theQuickTime format.

Stream data according to the present invention is made up of encodeddata included in a data file and auxiliary information included in anauxiliary information file. The encoded data is obtained by encodingvideo data and audio data in accordance with the MPEG-2 system standard,and is decodable by either the auxiliary information or the MPEG-2system standard. The auxiliary information includes: referenceinformation to make reference to the encoded data; and attributeinformation that uses a video object unit (VOBU) of the encoded data asa sample unit and that describes an attribute of the sample unit. Thisstream data is stored on a storage medium according to the presentinvention.

Another data processor according to the present invention includes: areading section for reading the auxiliary information file from thestream data and also reading the data file in response to a controlsignal; a reading control section for generating, as the control signal,a signal instructing that the data file be read in accordance with thereference information defined by the auxiliary information of theauxiliary information file; a decoding section, which receives theencoded data from the data file read and the auxiliary information andwhich decodes the encoded data into the video data and the audio data inaccordance with the attribute information included in the auxiliaryinformation; and an output section for outputting the video and audiodata decoded.

A data writing method according to the present invention includes stepsof: receiving video data and audio data; generating encoded data,complying with the MPEG-2 system standard, by encoding the video dataand audio data received; generating auxiliary information, whichincludes reference information to make reference to the encoded data andattribute information that uses a video object unit (VOBU) of theencoded data as a sample unit and that describes an attribute of thesample unit; and writing the encoded data and the auxiliary informationon a storage medium as a data file and an auxiliary information file,respectively. The encoded data is decodable by either the auxiliaryinformation file or the MPEG-2 system standard.

A data writing program according to the present invention is a computerprogram to be executed by a data processor. The data writing programincludes steps of: receiving video data and audio data; generatingencoded data, complying with the MPEG-2 system standard, by encoding thevideo data and audio data received; generating auxiliary information,which includes reference information to make reference to the encodeddata and attribute information that uses a video object unit (VOBU) ofthe encoded data as a sample unit and that describes an attribute of thesample unit; and writing the encoded data and the auxiliary informationon a storage medium as a data file and an auxiliary information file,respectively. The encoded data is decodable by either the auxiliaryinformation file or the MPEG-2 system standard.

A data reading method according to the present invention includes stepsof: reading the auxiliary information file from the stream data;generating a control signal instructing that the data file be read inaccordance with the reference information defined by the auxiliaryinformation of the auxiliary information file; reading the data file inresponse to the control signal; receiving the encoded data from the datafile read and the auxiliary information and decoding the encoded datainto the video data and the audio data in accordance with the attributeinformation included in the auxiliary information; and outputting thevideo and audio data decoded.

A data reading program according to the present invention is a computerprogram to be executed by a data processor. The data reading programincludes steps of: reading the auxiliary information file from thestream data; generating a control signal instructing that the data filebe read in accordance with the reference information defined by theauxiliary information of the auxiliary information file; reading thedata file in response to the control signal; receiving the encoded datafrom the data file read and the auxiliary information and decoding theencoded data into the video data and the audio data in accordance withthe attribute information included in the auxiliary information; andoutputting the video and audio data decoded.

Still another data processor according to the present inventionincludes: a receiving section for receiving video data and audio data; acompressing section, which encodes the received video and audio data bya predetermined encoding technique, thereby generating encoded data inwhich data representing the video data and data representing the audiodata are interleaved with each other; and an auxiliary informationgenerating section for generating auxiliary information, which includesreference information to make reference to the encoded data andattribute information that describes an attribute of a sample unit. Thesample is a set of the encoded data that has been collected according toa playback duration of the video data. The encoded data is decodable byeither the auxiliary information or a decoding technique correspondingto the predetermined encoding technique.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 shows a configuration for a conventional data processor 350.

FIG. 2 shows the data structure of an MP4 file 20.

FIG. 3 shows a specific example of the atom structure 23.

FIG. 4 shows the data structure of a moving picture stream 22.

FIG. 5 shows a moving picture stream 22 in which tracks are changed onthe way.

FIG. 6 shows a correlation between a moving picture stream 22 andsectors of a DVD-RAM disc 331.

FIG. 7 shows how the written data is managed by the file system of theDVD-RAM.

FIG. 8 schematically shows a correlation between the field names used asthe access information compliant with the DVD Video recording standardand pictures represented by the field names.

FIG. 9 shows the data structure of the access information shown in FIG.8, the field names defined for the data structure, and their contentsand data sizes.

FIG. 10 illustrates a connection environment for a portable videocorder10-1, a movie recorder 10-2 and a PC 10-3 for carrying out the dataprocessing of the present invention.

FIG. 11 shows an arrangement of functional blocks in a data processor10.

FIG. 12 shows the data structure of an MP4 stream 12 according to thepresent invention.

FIG. 13 shows the management unit of audio data in an MPEG2-PS 14.

FIG. 14 shows a correlation between a program stream and elementarystreams.

FIG. 15 shows the data structure of auxiliary information 13.

FIG. 16 shows the contents of respective atoms that make up an atomstructure.

FIG. 17 shows a specific exemplary description format for “DataReference Atom” 15.

FIG. 18 shows specific exemplary descriptions of respective atomsincluded in “Sample Table Atom” 16.

FIG. 19 shows a specific exemplary description format for “SampleDescription Atom” 17.

FIG. 20 shows the contents of respective fields of“sample_description_entry” 18.

FIG. 21 is a flowchart showing a procedure to generate the MP4 stream.

FIG. 22 is a table showing the differences between the MPEG2-PSgenerated by the processing of the present invention and a conventionalMPEG-2 Video (elementary stream).

FIG. 23 shows the data structure of the MP4 stream 12 in a situationwhere one VOBU is handled as one chunk.

FIG. 24 shows the data structure in the situation where one VOBU ishandled as one chunk.

FIG. 25 shows specific exemplary descriptions of respective atomsincluded in Sample Table Atom 19 in the situation where one VOBU ishandled as one chunk.

FIG. 26 shows an exemplary MP4 stream 12 in which two PS files areprovided for a single auxiliary information file.

FIG. 27 shows an example in which there are a number of discontinuousMPEG2-PS's within one PS file.

FIG. 28 shows an MP4 stream 12 in which a PS file, storing an MPEG2-PSfor the purpose of seamless connection, is provided.

FIG. 29 shows the audio frame that is absent from the discontinuitypoint.

FIG. 30 shows the data structure of an MP4 stream 12 according toanother example of the present invention.

FIG. 31 shows the data structure of an MP4 stream 12 according to stillanother example of the present invention.

FIG. 32 shows the data structure of an MTF file 32.

FIG. 33 shows a correlation among various types of file formatstandards.

FIG. 34 shows the data structure of a QuickTime stream.

FIG. 35 shows the contents of respective atoms in the auxiliaryinformation 13 of the QuickTime stream.

FIG. 36 shows the contents of flags defined for a moving picture streamin a situation where the number of recording pixels changes.

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, preferred embodiments of the present invention will bedescribed with reference to the accompanying drawings.

FIG. 10 illustrates how to connect a portable videocorder 10-1, a movierecorder 10-2 and a PC 10-3 for carrying out the data processing of thepresent invention.

The portable videocorder 10-1 receives a broadcast program via itsattached antenna and compresses the moving pictures of the broadcastprogram, thereby generating an MP4 stream. The movie recorder 10-2records not only video but also its accompanying audio, therebygenerating another MP4 stream. In an MP4 stream, the video and audiodata are encoded by a predetermined compression coding method and aredescribed in accordance with the data structure of the presentinvention. The portable videocorder 10-1 and movie recorder 10-2 eitherstore the generated MP-4 streams on a storage medium 131 such as aDVD-RAM or output the streams through a digital interface such as anIEEE 1394 or USB port. It should be noted that the portable videocorder10-1 and movie recorder 10-2 needs to have even smaller sizes. Thus, thestorage medium 131 does not have to be an optical disc with a diameterof 8 cm but may be an optical disc with a smaller diameter, for example.

The PC 10-3 receives the MP4 streams by way of either the storage mediumor a transmission medium. If the respective appliances are connectedtogether through a digital interface, then the PC 10-3 can receive theMP4 streams from the respective appliances by controlling the movierecorder 10-2 and so on as external storage devices.

If the PC 10-3 has application software or hardware that can cope withthe MP4 stream processing of the present invention, then the PC 10-3 canplay back the MP4 streams just as defined by the MP4 file standard. Onthe other hand, if the PC 10-3 cannot cope with the MP4 streamprocessing of the present invention, then the PC 10-3 can play back themoving picture streams in accordance with the MPEG-2 system standard. Itshould be noted that the PC 10-3 can also perform editing processingsuch as partial deletion on the MP4 streams. As used herein, the term“playback” includes such editing processing. In the followingdescription, the portable videocorder 10-1, movie recorder 10-2 and PC10-3 shown in FIG. 1 will be collectively referred to as a “dataprocessor”.

FIG. 11 shows an arrangement of functional blocks in a data processor10. In the following description, the data processor 10 is supposed tohave the capabilities of both reading and writing an MP4 stream. Morespecifically, the data processor 10 can not only generate an MP4 streamand write it on a storage medium 131 but also read an MP4 stream that isstored on the storage medium 131. The storage medium 131 may be aDVD-RAM disc, for example, and will be referred to herein as a “DVD-RAMdisc 131”.

First, the MP4 stream writing function of the data processor 10 will bedescribed. The data processor 10 includes a video signal input section100, an MPEG2-PS compressing section 101, an audio signal input section102, an auxiliary information generating section 103, a writing section120, an optical pickup 130 and a writing control section 141 asrespective components regarding this function.

The video signal input section 100 is implemented as a video signalinput terminal and receives a video signal representing video data. Theaudio signal input section 102 is implemented as an audio signal inputterminal and receives an audio signal representing audio data. Forexample, the video signal input section 100 and audio signal inputsection 102 of the portable videocorder 10-1 (see FIG. 10) may beconnected to the video output section and audio output section of atuner section (not shown) to receive a video signal and an audio signal,respectively. Also, the video signal input section 100 and audio signalinput section 102 of the movie recorder 10-2 (see FIG. 10) mayrespectively receive a video signal and an audio signal from the CCDoutput (not shown) and microphone output of a camera.

The MPEG2-PS compressing section (which will be simply referred toherein as a “compressing section”) 101 receives the video and audiosignals, thereby generating an MPEG-2 program stream (which will bereferred to herein as an “MPEG2-PS”) compliant with the MPEG-2 systemstandard. The MPEG2-PS generated may be decoded by itself in accordancewith the MPEG-2 system standard. The MPEG2-PS will be described infurther detail later.

The auxiliary information generating section 103 generates auxiliaryinformation for the MP4 stream. The auxiliary information includesreference information and attribute information. The referenceinformation is used to identify the MPEG2-PS that has been generated bythe compressing section 101 and may include the file name of theMPEG2-PS being written and its storage location on the DVD-RAM disc 131.On the other hand, the attribute information describes the attributes ofa sample unit of the MPEG2-PS. As used herein, the “sample” refers tothe minimum management unit in a sample description atom (to bedescribed later) as in the auxiliary information defined by the MP4 filestandard. The attribute information includes data size, playback timeand so on for each sample. One sample may be a data unit that can beaccessed at random, for example. In other words, the attributeinformation is needed to read the sample. Among other things, the sampledescription atom (to be described later) is sometimes called “accessinformation”.

Specific examples of the attribute information include the address ofthe data storage location, a time stamp representing playback timing, anencoding bit rate, and information about codec. The attributeinformation is provided for each of the video data and the audio data inevery sample. Except for the field description to be mentionedexplicitly soon, the attribute information complies with the contents ofauxiliary information for a conventional MP4 stream 20.

As will be described later, one sample according to the presentinvention is a single video object unit (VOBU) in the MPEG2-PS. Itshould be noted that “VOBU” refers to the video object unit as definedby the DVD Video recording standard. The auxiliary information will bedescribed in further detail later.

In accordance with the instruction given by the writing control section141, the writing section 120 controls the pickup 130, thereby writingdata at a particular location (i.e., address) on the DVD-RAM disc 131.More specifically, the writing section 120 writes the MPEG2-PS,generated by the compressing section 101, and the auxiliary information,generated by the auxiliary information generating section 103, on theDVD-RAM disc 131 as respectively different files.

The data processor 10 further includes a continuous data area detectingsection (which will be simply referred to herein as a “detectingsection”) 140 and a logical block management section (which will besimply referred to herein as a “management section”) 143 that operateduring the data write operation. In accordance with the instructiongiven by the writing control section 141, the continuous data areadetecting section 140 checks the availability of sectors, which aremanaged by the logical block management section 143, thereby detecting aphysically continuous unused area available. The writing control section141 instructs the writing section 120 to write the data on that unusedarea. A specific data writing method may be similar to that alreadydescribed with reference to FIG. 7, there is no particularly importantdifference, and the detailed description thereof will be omitted herein.It should be noted that the MPEG2-PS and the auxiliary information arewritten as separate files. Thus, their respective file names are writtenon the file identifiers shown in FIG. 7.

Hereinafter, the data structure of the MP4 stream will be described withreference to FIG. 12. FIG. 12 shows the data structure of an MP4 streamaccording to the present invention. The MP4 stream 12 includes anauxiliary information file (MOV001.MP4) including the auxiliaryinformation 13 and a data file (MOV001.MPG) of the MPEG2-PS 14 (whichwill be referred to herein as a “PS file”). A single MP4 stream is madeup of the data stored in these two files. In this description, the samename “MOV001” is given to the auxiliary information file and PS file toclearly indicate that these two files belong to the same MP4 stream butdifferent extensions are given to them. More specifically, the sameextension “MP4” as that of a conventional MP4 file is adopted as theextension of the auxiliary information file, while an extension “MPG”normally used in a conventional program stream is adopted as theextension of the PS file.

The auxiliary information 13 includes reference information dref to makereference to the MPEG2-PS 14 and further includes attribute informationthat describes the attributes of each video object unit (VOBU) of theMPEG2-PS 14. Since the attribute information describes the attributes ofeach VOBU, the data processor 10 can find a VOBU at any arbitrarylocation in the MPEG2-PS 14 on a VOBU basis and can perform playback andediting processing thereon.

The MPEG2-PS 14 is a moving picture stream, which is compliant with theMPEG-2 system standard and which is made up of video and audio packsthat are interleaved together. Each video pack includes a pack headerand encoded video data, while each audio pack includes a pack header andencoded audio data. In the MPEG2-PS 14, the data is managed on a videoobject unit (VOBU) basis, where a VOB includes moving picture data, eachunit of which has a length corresponding to a video playback duration of0.4 to 1 second. The moving picture data includes a plurality of videopacks and a plurality of audio packs. By reference to the informationdescribed in the auxiliary information 13, the data processor 10 canlocate and read any arbitrary VOBU. It should be noted that each VOBUincludes at least one GOP.

The MP4 stream 12 of the present invention is partly characterized inthat the MPEG2-PS 14 can be decoded not only by reference to theattribute information 13, which complies with the data structure of anMP4 stream as defined by the MPEG-4 system standard, but also inaccordance with the MPEG-2 system standard. The auxiliary informationfile and the PS file are stored separately, and therefore, the dataprocessor 10 can analyze and process them independently of each other.For example, an MP4 stream player, which can carry out the dataprocessing of the present invention, can adjust the playback duration ofthe MP4 stream 12 according to the attribute information 13, sense theencoding method of the MPEG2-PS 14 and decode it by its associateddecoding method. On the other hand, a conventional apparatus that candecode an MPEG2-PS may decode it in accordance with the MPEG-2 systemstandard. Thus, even any currently popular version of software orhardware, which is compliant with only the MPEG-2 system standard, canalso play back a moving picture stream included in the MP4 stream.

Optionally, not only the sample description atom on the VOBU basis butalso another sample description atom, which uses a number of frames ofthe audio data of the MPEG2-PS 14, corresponding to a predeterminedamount of time, as a management unit, may be provided as shown in FIG.13. The predetermined amount of time may be 0.1 second, for example. InFIG. 13, “V” denotes the video pack shown in FIG. 12 and “A” denotes theaudio pack. An audio frame corresponding to 0.1 second is made up of atleast one pack. As for AC-3, for example, one audio frame includes audiodata corresponding to 1,536 samples supposing the sampling frequency is48 kHz. In this case, the sample description atom may be provided eitherwithin a user data atom in a track atom or on an independent track. Inanother example, the auxiliary information 13 may use an audio frame,synchronized with a VOBU and corresponding to a duration of 0.4 secondto 1 second, as a unit and may store various attributes such as theoverall data size of the units, the data address of the top pack and atime stamp representing the output timing.

Next, the data structure of each video object unit (VOBU) of theMPEG2-PS 14 will be described. FIG. 14 shows a correlation between aprogram stream and elementary streams. In the MPEG2-PS 14, a single VOBUincludes a plurality of video packs V_PCK and a plurality of audio packsA_PCK. More exactly, a VOBU runs from a sequence header (i.e., SEQheader shown in FIG. 14) to a pack just before the next sequence header.That is to say, a sequence header is put at the top of each VOBU. On theother hand, the elementary stream (Video) includes a number N of GOPs,which include various types of headers (such as the sequence (SEQ)header and GOP header) and video data (including I-, P- and B-pictures).The elementary stream (Audio) includes a plurality of audio frames.

Each of the video and audio packs included in the VOBU of the MPEG2-PS14 is composed of the data included in its associated elementary stream(Video) or (Audio) so as to have a data size of 2 kilobytes. Asdescribed above, each pack is provided with a pack header.

It should be noted that if another elementary stream (not shown) isprovided for auxiliary video data such as subtitle data, then each VOBUof the MPEG2-PS 14 further includes packs of that auxiliary video data.

Next, the data structure of the auxiliary information 13 in the MP4stream 12 will be described with reference to FIGS. 15 and 16. FIG. 15shows the data structure of the auxiliary information 13. This datastructure is called an “atom structure” and has a layered architecture.For example, “Movie Atom” includes “Movie Header Atom”, “ObjectDescriptor Atom” and “Track Atom”, which is further subdivided into“Track Header Atom”, “Edit List Atom”, “Media Atom” and “User DataAtom”. A similar statement applies to the other Atoms shown in FIG. 15.

According to the present invention, the attributes of a sample unit aredescribed by using “Data Reference Atom (drf)” 15 and “Sample Table Atom(stbl)” 16, in particular. As described above, one sample corresponds toone video object unit (VOBU) of the MPEG2-PS. “Sample Table Atom” 16includes the six low-order atoms shown in FIG. 15.

FIG. 16 shows the contents of respective atoms that make up the atomstructure. “Data Reference Atom” stores the information identifying thefile of a moving picture stream (i.e., the MPEG2-PS) 14 in the form of aURL. On the other hand, “Sample Table Atom” describes the attributes ofrespective VOBUs with its low-order atoms. For example, “Decoding Timeto Sample Atom” stores the playback durations of the respective VOBUs.“Sample Size Atom” stores the data sizes of the respective-VOBUs. Also,“Sample Description Atom” shows that the PS file data making up the MP4stream 12 is the MPEG2-PS 14 and also provides detailed specificationsof the MPEG2-PS 14. In the following description, the informationdescribed by “Data Reference Atom” will be referred to herein as“reference information” and the information described by “Sample TableAtom” will be referred to herein as “attribute information”.

FIG. 17 shows a specific exemplary description format for “DataReference Atom” 15. The information identifying the file is described ina portion (“DataEntryUrlAtom” in this example) of the field describing“Data Reference Atom” 15. In this case, the file name and file storagelocation of the MPEG2-PS 14 are described as a URL. By reference to“Data Reference Atom” 15, the MPEG2-PS 14, which makes up the MP4 stream12 along with its auxiliary information 13, can be identified. It shouldbe noted that even before the MPEG2-PS 14 is written on the DVD-RAM disc131, the auxiliary information generating section 103 shown in FIG. 11can also detect the file name and file storage location of the MPEG2-PS14. This is because the file name can be determined in advance andbecause the file storage location can be logically identified by thenotation of the layered structure of the file system.

FIG. 18 shows specific exemplary descriptions of respective atomsincluded in “Sample Table Atom” 16. Each atom defines the field name,repeatability and data size. For example, “Sample Size atom” includesthree fields “sample-size”, “sample count” and “entry-size”. Among thesefields, the default data size of the VOBU is stored in the “sample-size”field, and an individual data size, which is different from the defaultvalue of the VOBU, is stored in the “entry-size” field. In the “setting”shown in FIG. 18, each parameter (such as “VOBU_ENT”) may have the samevalue as the access data of the same name according to the DVD Videostandard.

In “Sample Description Atom” 17 shown in FIG. 18, the attributeinformation of the sample unit is described. Hereinafter, the contentsof the information described in “Sample Description Atom” 17 will bedescribed.

FIG. 19 shows a specific exemplary description format for “SampleDescription Atom” 17. “Sample Description Atom” 17 describes its datasize and the attribute information of a sample unit when each VOBU is asingle sample. The attribute information is described in“sample_description_entry” 18 of “Sample Description Atom” 17.

FIG. 20 shows the contents of respective fields of“sample_description_entry” 18. The entry 18 includes “data format”specifying the encoding method of its associated MPEG2-PS 14. In FIG.20, “p2sm” shows that the MPEG2-PS 14 is an MPEG-2 program streamincluding MPEG-2 Video.

The entry 18 includes that sample's “Presentation Start Time” and“Presentation End Time”, which store the timing information of the firstvideo frame and the timing information of the last video frame,respectively. The entry 18 further includes the attribute information(“Video ES Attribute”) of the video stream within the sample and theattribute information (“Audio ES Attribute”) of the audio stream withinthe same sample. As shown in FIG. 19, the video data attributeinformation may define the CODEC type of the video (e.g., MPEG-2 Video)and the width and height (“width” and “height”) of the video data, forexample. In the same way, the audio data attribute information maydefine the CODEC type of the audio (e.g., AC-3), the number of channelsof the audio data (“channel count”), the size of the audio sample(“samplesize”) and the sampling rate thereof (“samplerate”).

The entry 18 further includes a discontinuity point start flag andseamless information. These pieces of information are described if thereare a number of PS streams in a single MP4 stream 12 as will bedescribed later. For example, a discontinuity point start flag of “0”indicates that the previous moving picture stream and the current movingpicture stream are a completely continuous program stream. On the otherhand, a discontinuity point start flag of “1” shows that those movingpicture streams are discontinuous program streams. If those streams arediscontinuous, the seamless information may be described in order toplay back a moving picture or audio without a break even at adiscontinuity point of the moving picture or audio. The seamlessinformation includes audio discontinuity information and SCRdiscontinuity information during the playback. The audio discontinuityinformation includes the presence or absence of a mute interval (i.e.,the audio gap shown in FIG. 31), the start timing and the time lengththereof. The SCR discontinuity information includes the SCR values ofthe two packs that are just before, and just after, the discontinuitypoint.

By providing the discontinuity point start flag, the switching point ofSample Description Entries and the continuity switching point of movingpicture streams can be defined independently of each other. As shown inFIG. 36, if the number of recording pixels changes on the way, thenSample Descriptions are changed. In this case, however, if the movingpicture streams themselves are continuous, then the discontinuity pointstart, flag may set to zero. If the discontinuity point start flag iszero, a PC, which is directly editing an information stream, canunderstand that seamless playback is realized even without resetting aconnection point between two moving picture streams. FIG. 36 shows asituation where the number of horizontal pixels has changed. However,the same technique is also applicable to a situation where any othertype of attribute information has changed. For example, a situationwhere a 4:3 aspect ratio has changed into 16:9 as to the aspectinformation or a situation where the audio bit rate has changed may alsobe coped with.

The data structures of the auxiliary information 13 and MPEG2-PS 14 ofthe MP4 stream 12 shown in FIG. 12 have been described. According to thedata structure described above, any portion of the MPEG2-PS 14 may bedeleted just by changing the attribute information (e.g., the timestamp) in the auxiliary information 13, and there is no need to changethe time stamp provided for the MPEG2-PS 14. Thus, the editing can bedone by taking advantage of a conventional MP4 stream. In addition,according to the data structure described above, if a moving picture isbeing edited on a PC with application or hardware compatible with astream compliant with the MPEG-2 system standard, just a PS file may beimported into the PC. This is because the MPEG2-PS 14 of the PS file isa moving picture stream compliant with the MPEG-2 system standard. Suchapplication or hardware has circulated widely, and therefore, any pieceof existent software or hardware can be used effectively. In addition,the auxiliary information can be stored in a data structure compliantwith the ISO standard.

Hereinafter, it will be described with reference to FIGS. 11 and 21 howthe data processor 10 generates an MP4 stream and writes it on a DVD-RAMdisc 131. FIG. 21 is a flowchart showing a procedure to generate the MP4stream. First, in Step 210, the data processor 10 receives video datathrough the video signal input section 100 and audio data through theaudio signal input section 102, respectively. Next, in Step 211, thecompressing section 101 encodes the received video and audio data inaccordance with the MPEG-2 system standard. Subsequently, in Step 212,the compressing section 101 makes up an MPEG2-PS of the video and audioencoded streams (see FIG. 14).

Then, in Step 213, the writing section 120 determines the file name andstorage location of the MPEG2-PS to be written on the DVD-RAM disc 131.Next, in Step 214, the auxiliary information generating section 103acquires the file name and storage location of the PS file and specifiesthe contents to be described as the reference information (i.e., DataReference Atom shown in FIG. 17). As shown in FIG. 17, a descriptionmethod that makes it possible to specify the file name and the storagelocation at the same time is adopted herein.

Subsequently, in Step 215, the auxiliary information generating section103 acquires data representing the playback duration, data size and soon for each of the VOBUs defined in the MPEG2-PS 14 and specifies thecontents to be described as the attribute information (i.e., SampleTable Atom shown in FIGS. 18 through 20). By providing the attributeinformation for each VOBU, any arbitrary VOBU can be read and decoded.This means that one VOBU is handled as one sample.

Thereafter, in Step 216, the auxiliary information generating section103 generates the auxiliary information based on the referenceinformation (i.e., Data Reference Atom) and the attribute information(i.e., Sample Table Atom).

Next, in Step 217, the writing section 120 outputs the auxiliaryinformation 13 and MPEG2-PS 14 as the MP4 stream 12 and writes them onthe DVD-RAM disc 131 as an auxiliary information file and a PS file,respectively. By performing this procedure, the MP4 stream is generatedand written on the DVD-RAM disc 131.

Hereinafter, the MP4 stream reading function of the data processor 10will be described with reference to FIGS. 11 and 12. On the DVD-RAM disc131, the MP4 stream 12, including the auxiliary information 13 andMPEG2-PS 14 having the data structures described above, is supposed tobe stored. Upon a user's request, the data processor 10 reads anddecodes the MPEG2-PS 14 that is stored on the DVD-RAM disc 131. The dataprocessor 10 includes a video signal output section 110, an MPEG2-PSdecoding section 111, an audio signal output section 112, a readingsection 121, the pickup 130 and a reading control section 142 asrespective components realizing the reading function.

First, in accordance with an instruction given by the reading controlsection 142, the reading section 121 controls the pickup 130 so as toread the MP4 file from the DVD-RAM disc 131 and acquire the auxiliaryinformation 13. The reading section 121 outputs the acquired auxiliaryinformation 13 to the reading control section 142. Also, in response toa control signal supplied from the reading control section 142 to bedescribed later, the reading section 121 reads the PS file from theDVD-RAM disc 131. The control signal is a signal designating the PS fileto read (“MOV001.MPG”).

The reading control section 142 receives the auxiliary information 13from the reading section 121 and analyzes its data structure, therebyacquiring the reference information 15 (see FIG. 17) contained in theauxiliary information 13. Then, the reading control section 142 outputsa control signal instructing that the PS file (“MOV001.MPG”) designatedby the reference information 15 be read from the specified location(i.e., “./” or root directory).

The MPEG2-PS decoding section 111 receives the MPEG2-PS 14 and theauxiliary information 13 and decodes the MPEG2-PS 14 into video data andaudio data in accordance with the attribute information contained in theauxiliary information 13. More specifically, the MPEG2-PS decodingsection 111 reads the data format (“data_format”), the video streamattribute information (“video ES attribute”) and the audio streamattribute information (“audio ES attribute”) of Sample Description Atom17 (see FIG. 19), and decodes the video and audio data in accordancewith the encoding method, the presentation size of the video data andthe sampling frequency as defined by those pieces of information.

The video signal output section 110 is implemented as a video signaloutput terminal to output the decoded video data as a video signal,while the audio signal output section 112 is implemented as an audiosignal output terminal to output the decoded audio data as an audiosignal.

The MP4 stream reading process by the data processor 10 begins byreading a file with an extension “MP4” (i.e., “MOV001.MP4”) as in theconventional process of reading an MP4 stream file. More specifically,this process may be carried out in the following manner. First, thereading section 121 reads the auxiliary information file (“MOV001.MP4”).Next, the reading control section 142 analyzes the auxiliary information13, thereby extracting the reference information (i.e., Data ReferenceAtom). Then, in accordance with the reference information extracted, thereading control section 142 outputs a control signal instructing thatthe PS file, making up the same MP4 stream, be read. In this preferredembodiment, the control signal output by the reading control section 142instructs that the PS file (“MOV001.MPG”) be read.

Next, in response to the control signal, the reading section 121 readsthe designated PS file. Thereafter, the MPEG2-PS decoding section 111receives the MPEG2-PS 14 and auxiliary information 13 contained in thedata file read and analyzes the auxiliary information 13, therebyextracting the attribute information. Then, by reference to SampleDescription Atom 17 (see FIG. 19) included in the attribute information,the MPEG2-PS decoding section 111 identifies the data format of theMPEG2-PS 14 (“data_format”), the attribute information of the videostream included in the MPEG2-PS 14 (“video ES attribute”) and theattribute information of the audio stream (“audio ES attribute”),thereby decoding the video data and audio data. By performing theseprocessing steps, the MPEG2-PS 14 is read in accordance with theauxiliary information 13.

It should be noted that any conventional player or playback softwarethat can read a stream compliant with the MPEG-2 system standard canread the MPEG2-PS 14 just by reading the PS file. In that case, however,the player does not have to be able to read the MP4 stream 12. Since theMP4 stream 12 is made up of the auxiliary information 13 and MPEG2-PS 14as two separate files, the PS file in which the MPEG2-PS 14 is storedcan be easily identified by the extension, for example, and read.

FIG. 22 is a table showing the differences between the MPEG2-PSgenerated by the processing of the present invention and a conventionalMPEG-2 Video (elementary stream). In FIG. 22, the column “the presentinvention (1)” summarizes the above example in which one VOBU is handledas one sample. In the conventional example, one video frame is handledas one sample and attribute information (access information) such asSample Table Atom is provided for each sample. In contrast, according tothe present invention, a VOBU including a plurality of video frames isused as a sample unit and the access information is provided for eachsample, thus cutting down the amount of attribute informationsignificantly. That is why one VOBU is preferably treated as one sampleas in the present invention.

In FIG. 22, the column “the present invention (2)” shows a modifiedexample of the data structure of “the present invention (1)”. Thedifference between “the present invention (2)” and “the presentinvention (1)” lies in that in this modified example (i.e., the presentinvention (2)), one VOBU corresponds to one chunk and that the accessinformation is defined on a chunk-by-chunk basis. As used herein, one“chunk” is a unit consisting of a plurality of samples. In this example,a video frame including the pack header of the MPEG2-PS 14 correspondsto one sample. FIG. 23 shows the data structure of the MP4 stream 12 ina situation where one VOBU is handled as one chunk. The difference isthat each sample shown in FIG. 12 is replaced by one chunk. In theconventional example, one video frame is handled as one sample and oneGOP is treated as one chunk.

FIG. 24 shows the data structure in the situation where one VOBU ishandled as one chunk. Comparing this data structure with that shown inFIG. 15 in which one VOBU is treated as one sample, it can be seen thatthe contents defined by Sample Table Atom 19 included in the attributeinformation of the auxiliary information 13 are different. FIG. 25 showsspecific exemplary descriptions of respective atoms included in SampleTable Atom 19 in the situation where one VOBU is handled as one chunk.

Hereinafter, a modified example of the PS file to make up the MP4 stream12 will be described. FIG. 26 shows an exemplary MP4 stream 12 in whichtwo PS files (“MOV001.MPG” and “MOV002.MPG”) are provided for a singleauxiliary information file (“MOV001.MP4”). In these two PS files, thedata of the MPEG2-PS 14, representing mutually different moving picturescenes, are stored separately. Within each PS file, the moving picturestream is continuous, and each of a system clock reference (SCR), apresentation time stamp (PTS) and a decoding time stamp (DTS), which areall compliant with the MPEG-2 system standard, is continuous, too.However, the SCR's, PTS's and DTS's are not continuous with each otherbetween the two PS files (i.e., between the end of MPEG2-PS #1 includedin one PS file and the beginning of MPEG2-PS #2 included in the other PSfile). These two PS files are treated as separate tracks (diagrams).

In the auxiliary information file, reference information (dref, see FIG.17) for identifying the file names and storage locations of therespective PS files is described. The reference information may bedescribed in the order of items to be referred to, for example. In FIG.26, the PS file “MOV001.MPG” identified by Reference #1 is read first,and then the PS file “MOV002.MPG” identified by Reference #2 is read.Even if there are a number of PS files in this manner, those PS filescan be read substantially continuously by providing referenceinformation for the respective PS files within the auxiliary informationfile.

FIG. 27 shows an example in which there are a number of discontinuousMPEG2-PS's within one PS file. In the PS file, MPEG2-PS's #1 and #2,representing different moving picture scenes, are arranged back to back.The “discontinuous MPEG2-PS's” mean that the SCR's, PTS's and DTS's arenot continuous with each other between the two MPEG2-PS's (i.e., betweenthe end of MPEG2-PS #1 and the beginning of MPEG2-PS #2). In otherwords, it means that the read timings are not continuous with eachother. The discontinuity point is located in the boundary between thetwo MPEG2-PS's. It should be noted that within each MPEG2-PS, the movingpicture stream is continuous and each of SCR, PTS and DTS, which are allcompliant with the MPEG-2 system standard, is continuous, too.

In the auxiliary information file, reference information (dref, see FIG.17) for identifying the file name and storage location of the PS file isdescribed. A single piece of reference information designating the PSfile is stored in the auxiliary information file. However, if that PSfile were read sequentially, then the read operation would stop at thediscontinuity point between MPEG2-PS #1 and MPEG2-PS #2 because theSCR's, PTS's and DTS's are discontinuous with each other there. Thus,information about this discontinuity point (e.g., location information(or address) of the discontinuity point) is described in the auxiliaryinformation file. More specifically, the location information of thediscontinuity point is stored as the “discontinuity point start flag”shown in FIG. 19. For example, during the read operation, the readingcontrol section 142 detects the location information of thediscontinuity point and reads the video data of MPEG2-PS #2, which islocated after the discontinuity point, in advance, thereby controllingthe read operation such that at least the video data can be played backwithout a break.

A procedure of reading two PS files, storing mutually discontinuousMPEG2-PS's, by providing two pieces of reference information for thefiles has been described with reference to FIG. 26. Optionally, as shownin FIG. 28, another PS file, storing an MPEG2-PS for the purpose ofseamless connection, may be newly inserted between the two PS files suchthat the two original PS files can be read seamlessly. FIG. 28 shows anMP4 stream 12 in which a PS file (“MOV002.MPG”), storing an MPEG2-PS forthe purpose of seamless connection, is provided. The PS file(“MOV002.MPG”) includes an audio frame that is absent from thediscontinuity point between MPEG2-PS #1 and MPEG2-PS #3. This point willbe described in further detail with reference to FIG. 29.

FIG. 29 shows the audio frame that is absent from the discontinuitypoint. In FIG. 29, the PS file storing MPEG2-PS #1 is identified by “PS#1” and the PS file storing MPEG2-PS #3 is identified by “PS #3”.

Suppose the data of PS #1 is processed first, and then that of PS #3 isprocessed. The DTS video frame on the second row and the PTS video frameon the third row represent time stamps of a video frame. As can be seenfrom these time stamps, PS files #1 and #3 can be played back withoutdiscontinuing the video. As to an audio frame, however, there is a muteinterval, in which no data is present for a certain period of time,after PS #1 has been played and before PS #3 starts being played. Withsuch an interval left, seamless playback could not be achieved.

Thus, PS #2 is newly provided and a PS file, including an audio framefor the purpose of seamless connection, is provided such that theauxiliary information file can make reference to that file. This audioframe includes audio data to fill the mute interval. For example, theaudio data that was written synchronously with the end of the movingpicture of PS #1 is copied. As can be seen from the audio frame rowshown in FIG. 29, the audio frame for the purpose of seamless connectionis inserted next to PS #1. The audio frame of PS #2 lasts until lessthan one frame before PS #3 begins. Accordingly, another piece ofreference information (dref shown in FIG. 28) to make reference to thisnew PS #2 is provided for the auxiliary information 13 and is definedsuch that PS #2 is referred to after PS #1.

In FIG. 29, no data interval for less than one audio frame (i.e., a muteinterval) is shown as “audio gap”. Alternatively, the mute interval maybe eliminated by adding extra data for one more audio frame to PS #2. Inthat case, PS #2 and PS #3 will include a portion with the same audiodata sample, i.e., a portion in which the audio frames overlap with eachother. Even so, no serious problem should arise. This is because as tothe overlapping portion, the same sound will be output no matter whichdata is read.

By performing these processing steps, even a number of discontinuous PSfiles can be read and decoded continuously with no time gap left.

In the example shown in FIG. 29, all PS files are supposed to bereferred to by the reference information dref. However, just the PS #2file may be referred to by any other atom (e.g., a uniquely defineddedicated atom) or the second PS track. In other words, only the PSfiles compliant with the DVD Video recording standard may be referred toby the dref atom. Alternatively, the audio frame in the PS #2 file maybe stored as an independent file for the elementary stream, may bereferred to by an independent audio track atom provided within theauxiliary information file, and may be described in the auxiliaryinformation file so as to be played back in parallel with the end of PS#1. The timing of playing back PS #1 and the audio elementary streamsimultaneously may be specified by Edit List Atom (see FIG. 15, forexample) in the auxiliary information.

In the preferred embodiments described above, the moving picture streamis supposed to be an MPEG-2 program stream. Alternatively, the movingpicture stream may also be an MPEG-2 transport stream (which will bereferred to herein as an “MPEG2-TS”) as defined by the MPEG-2 systemstandard.

FIG. 30 shows the data structure of an MP4 stream 12 according toanother example of the present invention. The MP4 stream 12 includes anauxiliary information file (“MOV001.MP4”), storing auxiliary information13, and the data file (“MOV001.M2T”) of an MPEG2-TS 14 (which will bereferred to herein as a “TS file”).

As in the MP4 stream 12 shown in FIG. 12, the TS file is also referredto by the reference information dref in the auxiliary information 13 inthis MP4 stream 12.

A time stamp is added to the MPEG2-TS 14. More specifically, in thisMPEG2-TS 14, a time stamp of 4 bytes to be referred to at the time oftransmission is additionally provided before a transport packet (whichwill be referred to herein as a “TS packet”) of 188 bytes. Accordingly,a TS packet containing video (V_TSP) and a TS packet containing audio(A_TSP) are each made up of 192 bytes. It should be noted that the timestamp may be provided behind the TS packet.

In the MP4 stream 12 shown in FIG. 30, the attribute information may bedescribed in the auxiliary information 13 with a TS packet, containingvideo data corresponding to a video playback duration of about 0.4second to about 1 second, treated as one sample as in the VOBU shown inFIG. 12. In addition, as in FIG. 13, the data size, data address andplayback timing of one frame of audio data may also be described in theauxiliary information.

Alternatively, one frame may be handled as one sample, and a pluralityof frames may be treated as one chunk. FIG. 31 shows the data structureof an MP4 stream 12 according to still another example of the presentinvention. In this case, if a plurality of TS packets, each containingvideo data corresponding to a video playback duration of about 0.4second to about 1 second, is handled as one chunk and if accessinformation is defined on a chunk-by-chunk basis as in FIG. 23, thenquite the same effects as those achieved by the MP4 stream 12 shown inFIG. 12 are also accomplished.

Even if the data structure shown in FIG. 30 or 31 is adopted, thearrangement of respective files and the processing to be carried outbased on the data structure are similar to those already described withreference to FIGS. 12, 13 and 23. Thus, the description thereof will beomitted herein because it is easily understandable just by applying thestatements about the video and audio packs shown in FIGS. 12, 13 and 23to the video TS packet with the time stamp (V_TSP) and the audio TSpacket with the time stamp (A_TSP) shown in FIG. 30.

Next, the file structure of another data format, to which the dataprocessing described above is also applicable, will be described withreference to FIG. 32. FIG. 32 shows the data structure of an MTF file32. The MTF 32 is a file for storing a written or edited moving picture.The MTF file 32 includes a plurality of continuous MPEG2-PS's 14, whileeach MPEG2-PS 14 includes a plurality of samples (“P2Sample”). Everysample (“P2Sample”) is one continuous stream. For example, as alreadydescribed with reference to FIG. 12, the attribute information may bedefined on a sample basis. In the foregoing description, this sample(“P2 Sample”) corresponds to a VOBU. Every sample includes a pluralityof video and audio packs, each of which contains a constant quantity ofdata of 2,048 bytes. Also, if two MTFs are combined together, then theresultant MTF will consist of at least two P2Stream's.

In the MTF 32, if two adjacent MPEG2-PS's 14 are one continuous programstream, then a single piece of reference information may be provided forthe continuous range, thereby making up one MP4 stream. On the otherhand, if two adjacent MPEG2-PS's 14 are a discontinuous program stream,then the data address of the discontinuity point may be included in theattribute information as shown in FIG. 27, thereby making up another MP4stream 12. Thus, the data processing described above is applicable tothe MTF 32, too.

It has been described how to handle an MPEG-2 system stream by extendingthe MP4 file format that was standardized in 2001. Alternatively,according to the present invention, the MPEG-2 system stream may also behandled even by extending the QuickTime file format or the ISO BaseMedia file format, too. This is because most of the specifications ofthe MP4 file format and the ISO Base Media file format are defined basedon, and have the same contents as, the QuickTime file format. FIG. 33shows a correlation among various types of file format standards. For atype of atom (moov, mdat) in which “the present invention”, “MP4 (2001)”and “QuickTime” overlap with each other, the data structure of thepresent invention described above can be adopted. As already described,the atom type “moov” is shown in FIG. 15 and other drawings as “MovieAtom” of the highest-order layer of the auxiliary information.

FIG. 34 shows the data structure of a QuickTime stream. The QuickTimestream also consists of a file (“MOV001.MOV”) describing the auxiliaryinformation 13 and a PS file (“MOV001.MPG”) including the MPEG2-PS 14.Compared with the MP4 stream 12 shown in FIG. 15, “Movie Atom” definedby the auxiliary information 13 of the QuickTime stream is partiallychanged. Specifically, “Null Media Header Atom” is replaced with “BaseMedia Header Atom” 36 newly provided, and “Object Descriptor Atom” shownon the third row of FIG. 15 is deleted from the auxiliary information 13shown in FIG. 34. FIG. 35 shows the contents of respective atoms in theauxiliary information 13 of the QuickTime stream. If the data of asample (VOBU) is neither a video frame nor an audio frame, the “BaseMedia Header Atom” 36 added indicates that. The other atom structureshown in FIG. 35 and its contents are the same as those of the MP4stream 12 described above and the description thereof will be omittedherein.

In the preferred embodiments of the present invention described above,the MPEG2-PS 14 shown in FIG. 12 is supposed to contain moving picturedata (VOBU) for 0.4 second to 1 second. However, the time range may bedifferent. Also, the MPEG2-PS 14 is supposed to consist of VOBUscompliant with the DVD Video recording standard. Alternatively, theMPEG2-PS 14 may also be a program stream compliant with the MPEG-2system standard or a program stream compliant with the DVD Videostandard.

In the data processor 10 shown in FIG. 11, the storage medium 131 issupposed to be a DVD-RAM disc. However, the storage medium is notparticularly limited to a DVD-RAM. Examples of other preferred storagemedia 131 include optical storage media such as an MO, a DVD-R, aDVD-RW, a DVD+RW, a CD-R and a CD-RW and magnetic recording media suchas a hard disk. As another alternative, the storage medium 131 may alsobe a semiconductor storage medium such as a semiconductor memory.

The data processor 10 performs the processing of generating, writing andreading a data stream according to a computer program. For example, theprocessing of generating and writing the data stream may be carried outby executing a computer program that is described based on the flowchartshown in FIG. 21. The computer program may be stored in any of varioustypes of storage media. Examples of preferred storage media includeoptical storage media such as optical discs, semiconductor storage mediasuch as an SD memory card and an EEPROM, and magnetic recording mediasuch as a flexible disk. Instead of using such a storage medium, thecomputer program may also be downloaded via a telecommunications line(e.g., through the Internet, for example) and installed in the opticaldisc drive 100.

INDUSTRIAL APPLICABILITY

According to the present invention, while the data structure ofauxiliary information is adapted to the updated standard of today so asto comply with the ISO standard, a data structure for a data stream,having a format equivalent to a conventional one, and a data processor,operating on such a data structure, are provided as well. Since the datastream is compatible with the conventional format, any existentapplication can use the data stream. Consequently, every piece ofexistent software and hardware can be used effectively.

1. A data processor comprising: a receiving section for receiving videodata and audio data; a compressing section for generating encoded data,complying with the MPEG-2 system standard, by encoding the video dataand the audio data received; an auxiliary information generating sectionfor generating auxiliary information, which includes referenceinformation to make reference to the encoded data and attributeinformation that uses a video object unit (VOBU) of the encoded data asa sample unit and that describes an attribute of the sample unit; and awriting section for writing the encoded data and the auxiliaryinformation on a storage medium as a data file and an auxiliaryinformation file, respectively, wherein the encoded data is decodable byeither the auxiliary information file or the MPEG-2 system standard. 2.The data processor of claim 1, wherein the reference informationrepresents the file name and storage location of the data file stored onthe storage medium.
 3. The data processor of claim 1, wherein thecompressing section generates the encoded data as a plurality of sets,and wherein the auxiliary information generating section generates thereference information that makes reference to each said set of encodeddata.
 4. The data processor of claim 1, wherein the compressing sectiongenerates the encoded data as a plurality of sets, and wherein theauxiliary information generating section generates stream data as asingle stream by arranging the plurality of sets of encoded data as aseries, and also generates auxiliary information that further describeslocation information specifying the storage location of the encoded dataif the data size of the encoded data is not constant every time the datais read.
 5. The data processor of claim 1, wherein the compressingsection generates the encoded data as either an MPEG-2 program stream oran MPEG-2 transport stream.
 6. The data processor of claim 1, whereinthe auxiliary information generating section describes an audio frame ofencoded audio data, representing the audio data of the encoded data, asanother sample unit in the attribute information.
 7. The data processorof claim 3, wherein the compressing section generates first, second andthird data files, the second data file including frame data that isneeded to decode the encoded data of the first and third data filescontinuously with no time gap left.
 8. The data processor of claim 1,wherein the auxiliary information generating section generates anauxiliary information file that is described in the MP4 format.
 9. Thedata processor of claim 1, wherein the auxiliary information generatingsection generates an auxiliary information file that is described in theQuickTime format.
 10. Stream data comprising: encoded data included in adata file; and auxiliary information included in an auxiliaryinformation file, wherein the encoded data is obtained by encoding videodata and audio data in accordance with the MPEG-2 system standard, andis decodable by either the auxiliary information or the MPEG-2 systemstandard, and wherein the auxiliary information includes: referenceinformation to make reference to the encoded data; and attributeinformation that uses a video object unit (VOBU) of the encoded data asa sample unit and that describes an attribute of the sample unit.
 11. Astorage medium on which the stream data of claim 10 is stored.
 12. Adata processor comprising: a reading section for reading the auxiliaryinformation file from the stream data of claim 10 and also reading thedata file in response to a control signal; a reading control section forgenerating, as the control signal, a signal instructing that the datafile be read in accordance with the reference information defined by theauxiliary information of the auxiliary information file; a decodingsection, which receives the encoded data from the data file read and theauxiliary information and which decodes the encoded data into the videodata and the audio data in accordance with the attribute informationincluded in the auxiliary information; and an output section foroutputting the video and audio data decoded.
 13. A data writing methodcomprising steps of: receiving video data and audio data; generatingencoded data, complying with the MPEG-2 system standard, by encoding thevideo data and audio data received; generating auxiliary information,which includes reference information to make reference to the encodeddata and attribute information that uses a video object unit (VOBU) ofthe encoded data as a sample unit and that describes an attribute of thesample unit; and writing the encoded data and the auxiliary informationon a storage medium as a data file and an auxiliary information file,respectively, wherein the encoded data is decodable by either theauxiliary information file or the MPEG-2 system standard.
 14. A computerprogram to be executed by a data processor, the program comprising stepsof: receiving video data and audio data; generating encoded data,complying with the MPEG-2 system standard, by encoding the video dataand audio data received; generating auxiliary information, whichincludes reference information to make reference to the encoded data andattribute information that uses a video object unit (VOBU) of theencoded data as a sample unit and that describes an attribute of thesample unit; and writing the encoded data and the auxiliary informationon a storage medium as a data file and an auxiliary information file,respectively, wherein the encoded data is decodable by either theauxiliary information file or the MPEG-2 system standard.
 15. A datareading method comprising steps of: reading the auxiliary informationfile from the stream data of claim 10; generating a control signalinstructing that the data file be read in accordance with the referenceinformation defined by the auxiliary information of the auxiliaryinformation file; reading the data file in response to the controlsignal; receiving the encoded data from the data file read and theauxiliary information and decoding the encoded data into the video dataand the audio data in accordance with the attribute information includedin the auxiliary information; and outputting the video and audio datadecoded.
 16. A computer program to be executed by a data processor, theprogram comprising steps of: reading the auxiliary information file fromthe stream data of claim 10; generating a control signal instructingthat the data file be read in accordance with the reference informationdefined by the auxiliary information of the auxiliary information file;reading the data file in response to the control signal; receiving theencoded data from the data file read and the auxiliary information anddecoding the encoded data into the video data and the audio data inaccordance with the attribute information included in the auxiliaryinformation; and outputting the video and audio data decoded.
 17. A dataprocessor comprising: a receiving section for receiving video data andaudio data; a compressing section, which encodes the received video andaudio data by a predetermined encoding technique, thereby generatingencoded data in which data representing the video data and datarepresenting the audio data are interleaved with each other; and anauxiliary information generating section for generating auxiliaryinformation, which includes reference information to make reference tothe encoded data and attribute information that describes an attributeof a sample unit, wherein the sample is a set of the encoded data thathas been collected according to a playback duration of the video data,and wherein the encoded data is decodable by either the auxiliaryinformation or a decoding technique corresponding to the predeterminedencoding technique.